The Renormalization Group Studies on Four Fermion Interaction Instabilities on 
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We study the instabilities caused by four fermion interactions on algebraic spin liquids. Renor- 
malization group (RG) is used for three types of previously proposed spin liquids on the square 
lattice: the staggered flux state of SU(2) spin system, the 7r— flux state of SU(4) spin system, and 
the 7T— flux state of SU(2) spin system. The low energy field theories of the first two types of spin 
liquids are QED3 with emerged SU(4) and SU(8) flavor symmetries , the low energy theory of the 
7r-flux SU(2) spin liquid is the QCD3 with SU(2) gauge field and emergent Sp(4) (SO(5)) flavor 
symmetry. Suitable large-iV generalization of these spin liquids are discussed, and a systematic 
1/N expansion is applied to the RG calculations. The most relevant four fermion perturbations are 
identified, and the possible phases driven by relevant perturbations are discussed. 



I. INTRODUCTION 



Many algebraic spin liquid states have been proposed 
in 2+1 dimensional strongly correlated electronic sys- 
tems. In these spin liquids neither the spin rotation sym- 
metry nor the spatial discrete symmetry is broken, and 
the physical order parameters have algebraic correlations. 
The gapless excitations of the system include fractional- 
ized spin excitations (the spinon) which are usually cen- 
tered around isolated Dirac points, and in many cases 
also gapless gauge bosons. It is believed that when the 
number of gapless spinons with Dirac fermion spectrum 
is small enough, or when all the spinons are gapped, the 
gauge fields are confining. However, with large enough 
fermion numbers N, the system is believed to be de- 
scribed by a conformal field theory (CFT), with the fixed 
point gauge field coupling e* 2 ~ 1/N. Physics based on 
this conformal field theory at large- N case has been stud- 
ied in many references' 2 -^, and it has been shown that 
the order parameters of various spin ordered patterns 
with different symmetry breaking can be all described as 
fermion bilinears at these critical spin liquids. 

Reference^ has provided us with a general formalism of 
studying the algebraic spin liquids. For spin-1/2 system, 
the lattice mean field variational Hamiltonian enjoys a 
SU(2) local gauge symmetry, on top of spin SU(2) global 
symmetry and all the lattice symmetries. The specific 
type of gauge symmetry that survives at low energy field 
theory depends on the choice of background mean field 
variational parameters, and the low energy gauge sym- 
metries can be SU(2), U(l), or even Z 2 . And since the 
Z 2 gauge field only introduces short range interaction be- 
tween slave particles, the low energy long distance physics 
will not be modified by Z 2 gauge field. Thus we will only 
consider spin liquids with SU(2) and U(l) gauge field. 
Three types of spin liquids are of particular interest to 
us. The first two are the so-called staggered flux state of 
SU(2) spin system, and the it— flux state of SU(4) spin 
system 6,7 . Both states are expected to be described by 



the following action 
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The ellipses include all the other terms allowed by the 
symmetry or generalized symmetry transformations of 
the system. 7^ with jj, = 1, 2, 3 are just three Pauli 
matrices, which is special for d = 2. Without the el- 
lipses, action (|TJ) describes a conformal field theory Spin 
liquids described by |T]) enjoy U(l) local gauge symme- 
try and SU(4N) flavor symmetry. For the staggered flux 
phase of SU(2) spin system, N = 1, and for the 7r— flux 
phase of SU(4) spin system N = 2. Since the SU(4N) fla- 
vor symmetry is larger than the physical symmetry, it is 
called the emergent flavor symmetry. As has been studied 
previously, all the fermion bilinears are forbidden by sym- 
metry and projective symmetry transformations (PSG) 1 , 
the only allowed local field theory terms which break the 
emergent flavor symmetry down to physical symmetries 
are four fermion interaction terms. Four fermion inter- 
actions violate the conformal invariance of actions JTJ), 
therefore it plays the role of instabilities of the CFT. In 
the limit of N — > +00, the scaling dimension of any four 
fermion terms is 4, which is obviously irrelevant. At fi- 
nite N, whether these four fermion terms are relevant or 
not can be studied by explicitly calculating the 1 /N cor- 
rections to the bare scaling dimension, and this will be 
one of the goals of the current paper. 

Since all the four fermion terms are scalar under all 
the physical symmetry transformations, they should be 
mixed under RG flow. For the U(l) spin liquids described 
by (HJ, we will consider three types of four fermion terms. 
The first type of four fermion terms will preserve the 
SU(4N) emergent flavor symmetry. Due to the Pauli ma- 
trices nature of the Dirac matrices 7 M , there are only two 
terms in this category, and they will mix at the first order 
of 1/N expansion. We will show that these four fermion 
terms are likely to be irrelevant for even very small N, 
i.e. they will not create any instability. The second type 
of four fermion terms will break the SU(4N) symmetry 
down to Sp(4N) symmetry, this perturbation alone will 
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be relevant at the CFT for small enough N, and it is 
likely that it will drive the system to another fixed point 
which describes a spin liquid with Sp(4N) symmetry and 
gapless U(l) gauge bosons^. At d = 2 there is only one 
term of this kind. The third type of four fermion terms 
break the flavor symmetry down to SU(2N) x SU(2), and 
we will show that there is also a relevant linear combi- 
nation. However with the presence of both the second 
and third type of four fermion terms, the symmetry of 
the system is only Sp(2N) x U(l). 

The physical meaning of these symmetry breaking can 
be understood as following: In the N = 1 case, the phys- 
ical symmetry is SU(2) (~ Sp(2)) spin symmetry plus all 
the lattice symmetries. It is also quite popular to inter- 
pret the four fold degenerate valence bond solid (VBS) 
states as an 0(2) vector with Z 4 anisotropy, and the Z4 
symmetry breaking is possibly irrelevant at the critical 
point between Neel and VBS phase o 9 ' 10 ? 11 . In the alge- 
braic spin liquid formalism in this work, the VBS states 
are interpreted as fermion bilinears, and indeed trans- 
form as a planar vector under the U(l) group generator 
/i z . The Z4 anisotropy of the 0(2) vector should in- 
volve very high order of fermion interactions which are 
negligible at the CFT. Thus at the end of the chain 
of symmetry breaking, the symmetry is Sp(2) x U(l), 
which is identical to the symmetry with the presence 
of both SU(2N) x SU(2) four fermion terms and Sp(4N) 
four fermion terms (N = 1). Thus driven by the 
SU(2N) x SU(2) terms, the Sp(4N) fixed point is sur- 
rounded by phases with smaller symmetries, some of the 
phases will break the Sp(2N) spin symmetry, and some 
other phases may break the U(l) symmetry (the enlarged 
discrete symmetry). Therefore the Sp(4N) fixed point is 
a critical point (or multicritical point) between phases 
breaking completely different symmetries. 

The N = 2 case corresponds to the ir— flux state of 
SU(4) spin system on the square lattice, and recent nu- 
merical results suggest that the ir— flux state is a good 
candidate of the ground state of SU(4) Heisenberg model 
on the square lattice^. The SU(4) spin and pseu- 
dospin symmetry have been discussed in spin-orbit cou- 
pled system a 13 ' 14 as well as spin-3/2 fcrmionic cold atom 
syste m 15 ' 16 i 17 . In spin-3/2 cold atom systems, since the 
particle density is very diluted, only the s— wave scat- 
tering should be considered. In this case, without fine- 
tuning any parameter, the system automatically enjoys 
Sp(4) (S0(5)) symmetry. And by tuning the ratio be- 
tween the spin-2 scattering channel and spin-0 scatter- 
ing channel, one can reach a critical point with SU(4) 
spin symmetry. In the spin-3/2 cold atom system at the 
vicinity of the SU(4) point, all the four fermion terms dis- 
cussed above should exist as a perturbation to the 7r— flux 
state. 

The third type of spin liquid we will discuss is the 
7T— flux state of SU(2) spin system. This state is invari- 
ant under SU(2) local gauge transformation even at low 



energy field theory^: 
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a 1 with I = 1,2, 3 are three Pauli matrices of the SU(2) 
gauge group. The flavor symmetry of this state has been 
shown to be Sp(4)^. However, the SU(2) gauge field for- 
malism makes the spin SU(2) symmetry inapparent£. In 
reference^, in order to make the SU(2) gauge symmetry 
and the SU(2) spin symmetry both apparent, the authors 
had to double the number of fermion components, but 
now the fermion multiplct suffers from a constraint: ip* 
and ip are related through a unitary transformation. In 
order to do calculations without constraint, in this paper 
we will first introduce a Major ana fermion formalism for 
this 7T— flux state. In this formalism there are eight com- 
ponents of Majorana fermions with 2 Dirac species each, 
and the system enjoys an SO (8) flavor symmetry in the 
absence of gauge fluctuations. The SU(2) ~ SO (3) gauge 
group as well as Sp(4) ~ SO (5) flavor symmetry group 
are both subgroups of the SO (8) group. The Neel and 
VBS order parameters still form a vector representation 
of the S0(5) flavor group. In the large- N generalization, 
the gauge group is still SU(2), and the flavor symmetry 
is Sp(2N), AT = 2™, n = 1, 2, The large-TV gen- 

eralization is applicable to the ir— flux state of Sp(2N) 
spin models with TV = 2™ -1 . Our calculations show that 
the SO (5) invariant four- fermion terms are not going to 
introduce any instability to the 7r— flux state, while the 
S0(5) breaking terms are relevant perturbations. 

Our large- TV calculations have used some algebras and 
identities of SU(N), Sp(2N) Lie Algebras. The detailed 
analysis of the group theory and algebras will be sum- 
marized in the appendix. In section II and III we will 
study U(l) and SU(2) spin liquids respectively. In our 
calculations 1/7V is the only small parameter used for ex- 
pansion, and we do not assume e = d — 1 to be small. 
Our loop integrals and field propagators are calculated 
in d — 2, and a rigorous e expansion should involve a 
general d calculations. However, at general dimensions 
there are many more four fermion terms than the d = 2 
case, simply because at d — 2 the three gamma matrices 
are Pauli matrices, the Fierz identity reduces the number 
of four fermion terms significantly, which is a very con- 
venient property we want to make full use of. A formal 
rigorous general d calculation is possible, we will leave it 
to the future study. 



II. SPIN LIQUIDS WITH U(l) GAUGE FIELD 

A. SU(4N) four fermion terms 

The low energy field theory of the staggered flux state 
of SU(2) spin system and the ir— flux state of SU(4) spin 
system are proposed to be described by CFT in equation 
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(fTj). Four fermion interactions are one type of instabili- 
ties. As has been mentioned in the introductory section, 
we will focus on three types of four fermion terms. The 
first type contains two terms: 



u = 
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Hereafter the bracket denotes the trace in the Dirac 
space. The number N and cut-off A at the denomina- 
tor is to guarantee both terms are at order of TV and the 
coefficients are dimensionless constants. In equation ([3]), 
a = 1, • ■ - AN is flavor indices, and we will focus on the 
case with N = 2"" 1 . 

Li and L 2 are the only two four fermion terms which 
are both SU(4N) and Lorentz invariant. Throughout 
the paper we will only consider four fermion terms with 
Lorentz invariance, partly because a large class of in- 
teresting quantum critical points are z = 1 theories 
with emergent Lorentz invariance; the Lorentz symme- 
try breaking effects in the kinetic terms of equation 
([T]) have been considered in reference^, and they were 
showed to be irrelevant. Several other SU(4N) invariant 
terms can be written down, for instance (ip a ' l l > b)('4 > b' l l > a)i 

(ip a 7i^ipb)(ipb7^a), Eili ~ X tyaT^iibWcTl^d) and 

EtV 2 ~\^alvT^ b )$ c ^ T M- Here T* are funda- 
mental representations of SU(4N) algebra. However, us- 
ing the Fierz identity of 7^ matrices, and the identity 
(|45|) in the appendix, all these terms can be written as 
linear combinations of L\ and L 2 . 

We will calculate the RG equation for the linear and 
quadratic order of the four fermion couplings. The first 
order corrections from 1 /N expansion will be calculated 
for the linear term, and for the quadratic terms only the 
leading order of unity is calculated. Notice that when 
5i = 9i = the system is at the CFT fixed point, so the 
point with zero four-fcrmion coupling is always a fixed 
point, despite the fact that gauge field fluctuations will 
generate effective four-fermion interactions^, the effects 
of these generated effective four-fermion interactions are 
included in diagram E and F of Fig. O At the CFT 
fixed point, the scaling dimensions of fermion bilinears 
have been calculated elsewherei^. For instance, 
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These two fermion bilinears belong to different represen- 
tations of the SU(4N) algebra, therefore their scaling di- 
mensions should in principle differ from each other. No- 
tice that the scaling dimension of conserved current tpl^ 
and ^7 /J T a , do not gain any corrections from the l/N 
expansion at this CFT fixed point, simply because the 
conservation law requires their scaling dimensions to be 
exactly 2. 

The l/N correction of scaling dimensions mainly 



FIG. 1: Feynman diagrams contribute to the linear orders 
in both Eqs. [6] and [9] The dashed lines are dressed photon 
propagators, and the full circles denote the trace in Dirac 
space. 





FIG. 2: Feynman diagrams which only contribute to the linear 
orders in Eqs. [6] but not in Eqs. [9] 



comes from the dressed photon propagator 
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The Feynman diagrams which contribute to both 171 and 
g[ are listed in Fig. [TJ Diagrams A and B are usu- 
ally called the vertex and wavefunction renormalizations, 
which also contribute to fermion bilinears. Besides these 
one-loop diagrams, there are also two-loop diagrams (Fig. 
[2]), which involve two photon propagators and one extra 
trace in the fermion flavor space, and hence also belongs 
to the l/N order correction. 

As already mentioned above, the quadratic terms in 
the equations are only calculated to the order of unity, 
the only Feynman diagram that contributes to this order 
is diagram G in Fig. [3l all the other diagrams will contain 
one extra 1/iV. After counting all the diagrams, the final 
RG equations are 
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FIG. 3: Feynman diagrams which contribute to the quadratic 
order of the RG equations (JSJ) and Notice that since we 
only calculate to the order of unity in the quadratic terms, 
diagram G only contributes to equation ((6} but not ©, and 
diagram H only contributes to equation @ but not (JSJ. 



FIG. 4: Feynman diagrams which contribute to the difference 
of scaling dimensions of fermion bilinears ^/>Ts'u(4N)/sp(4N)V , 
and V/T s a p(4N) T/>. 
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Here e = d — 1 = 1. At the fixed point gi — g[ — 
0, the largest eigenvalue of flowing equations is — 1 + 
1.39/ (47V), which is always negative for any integer TV. 
Thus we conclude that the The four fermion interactions 
which preserves SU(4N) symmetries are likely irrelevant 
for all TV. And no stable fixed point is found at finite 
four fermion coupling. Another way of interpreting this 
result is that, the flavor symmetry preserving mass gap 
is not generated spontaneously, which is consistent with 
reference^. 



B. Sp(4N) four fermion terms 

The second type of four fermion terms will break 
SU(4N) symmetry to Sp(4N) symmetry^. In Sp(4N) al- 
gebra there is a 4TV x 47V antisymmetric tensor j7a/3 which 
satisfy 

/jrpQ, rr / rjia \t 



JT" 



su(4N)/sp(4N) 



J 



frpa \t 
~V J su(4N)/sp(4N)J ' 
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T sp(4N) are elements of Sp(4N) algebra, and T s a u(4N)/sp(4N) 
are elements in SU(4N) algebra but not Sp(4N) algebra. 
All the algebra elements for TV = 2 n_1 have been con- 
structed in the appendix. The only four fermion term of 
this type is 



ff2 

47VA 



(8) 



The other current-current interaction term 
Ja 1 Jf3a(ipal^i3)(i>^ l iipa) actually equals L 2 if 
one uses the Fierz identity of Dirac gamma matrices in 
d = 2: 7q/377,t = 28 a(T 6p 7 - 5 a p5 ia . 

The Feynman diagrams in Fig. [5] do not contribute to 
g 2 , and the diagram H in Fig. [3] will contribute to the 
order of unity in the quadratic term in the RG equation: 
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This equation has fixed points at gi = and gi — g\ — 
37r 2 (-e + 64/(47V7r 2 )). At d = 2 and TV = 1 we now find a 
result which is very different from the SU(4N) perturba- 
tions above. The gi = fixed point is unstable with RG 
eigenvalue 0.621, while the fixed point at 52 = <?2 > is 
stable. Notice that the quadratic term in this equation 
is the only term with 0(1 /TV ) coefficient, all the other 
nonlinear terms gain 1/TV coefficient, thus the existence 
of this fixed point can be obtained from 1/TV expansion 
with TV extrapolating back to TV = 1, even without as- 
suming e to be small. All the higher order terms in the 
1/TV expansions will only move the critical point by or- 
der of 1/TV at most. Although now the fixed point value 
(?2 is of order unity, there is always a number 4TV at the 
denominator of 32, thus 52 /(47V) can still be treated per- 
turbatively close to the fixed point, as long as we do not 
encounter an extra factor of 47V in our calculation. Be- 
cause Li is an pair-pair interaction term, no extra factor 
of 47V is gained in our calculation if we only calculate the 
scaling dimensions of terms like ipTip. The correction of 
$2 to the scaling dimensions of L\ and L[ is also at the 
order of 1/(4 A). For TV = 2, to the order of expansion 
done here, the fixed point with zero four-fermion terms 
is stable against Li perturbation, and the finite four- 
fermion coupling fixed point become instable. However, 
higher order 1/TV corrections might change this result for 
TV = 2. Hereafter we will denote the critical value of TV 
as TV c i. One can also tune N close to 7V c i, and since g% 
is linear with (TV — 7V ci )/TV, at the vicinity of the critical 
TV, can be treated perturbatively. 

The scaling dimensions of all (4TV) 2 — 1 fermion bi- 
linears xpT a ip (T a are SU(4N) generators) equal at the 
fixed point with gi = which preserves the SU(4N) sym- 
metry. The difference between the scaling dimensions of 
tjjT a ip and iptp is from the diagrams similar to the ones in 
Fig. [3}^ with two photon propagators and a trace in the 
fermion flavor space, which only contributes to fermion 
bilinear tpip. At the Sp(4N) symmetric fixed point, the 
scaling dimensions of fermion bilinears are classified as 
the representation of Sp(4N) algebra: 'tptp and 'fpT^^^ip 
form scalar and adjoint representations of Sp(4N) group 



respectively, the appendix proved that ipT a 



(4N)/sp(4N) 



also form a representation of Sp(4N) group at least for 
TV = 2 n_1 . For instance, for the case with n = 1, 
SU(4)/Sp(4) are just five Gamma matrices, which form 
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a vector representation of SO(5) (Sp(4)) algebra. The 
scaling dimensions of fcrmion bilinears within the same 
representation are equal to each other. 

If we assume (N — N c \)/N and 1/N are of the same 
order, the scaling dimensions of the fermion bilinears at 
the Sp(4N) fixed point deviate from their value at the 
SU(4N) fixed point at the order of 1/N 2 , and requires 
a lot more calculations. But their differences at 1/N 2 
order can be calculated readily from diagrams in Fig. [4j 
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To obtain these results we have used the identities in 
equation ([7]). Without assuming N — N c i to be small, 
the scaling dimensions of fermion bilinears at the Sp(4N) 
fixed point can be calculated to the 1/N order as: 



A('0T su (4 N )/ S p( 4N )V>) = 2 — 

A(V>T sp(4N) V>) = 2 - 

A(^) = 2 + 
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Notice that the diagrams in Fig. [4] are the only two 
diagrams which can contribute to the 1/N order of the 
scaling dimensions of fermion bilinears. 

The Sp(4N) fixed point is located at the side with g 2 > 
0. At the Sp(4N) fixed point, the modified linear order 
RG equation for L\ and L[ reads: 
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Thus at this Sp(4N) fixed point the SU(4N) perturba- 
tions L\ and L\ are even more irrelevant compared with 
at the SU(4N) fixed point. 

When (?2 < there is no stable fixed point and when N 
is small enough the system will be driven to a state with 
only short range correlations. In this case the most fa- 
vored state is likely to be the Sp(4N) singlet pairing state. 
In general the pairing amplitude SaP^a^fij) — Ciji 
and Cij is a symmetric tensor, i, j are Dirac matrices 
indices. For convenience, we can choose the meanfield 
pairing amplitude to be •J a /3(ifia,i' l Pi3,j) = C<%; with con- 
stant C. This state breaks the U(l) gauge symmetry to 
Z 2 gauge symmetry because of the fermion pairing, the 
particle conservation of tp is also broken to conservation 
mod 2, but the Sp(4N) symmetry is preserved simply 
because the pairing is in the Sp(4N) singlet channel. 

Using the identities P5|) proved in the appendix, L 2 
can also be written as 
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The ellipses are SU(4N) invariant terms L\ and L[. Thus 
when g 2 is negative and grow large, the system may also 
develop order ($T*, 4 ^, ^'ij)) ^ 0, which breaks the 
Sp(4N) symmetry. The competition between the Sp(4N) 
symmetry breaking state and the Sp(4N) singlet pairing 
state requires further detailed analysis. 



C. SU(2N) x SU(2) four fermion terms 
The third type of four fermion terms are 



u = 
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L 3 = ^^(VW7^a&)(VW^/3a)- (14) 

Here a and j3 are indices in the SU(2N) subspace, and a 
and b are indices in the SU(2) space. These two terms 
have other representations using identities proved in the 
appendix: 
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Again the ellipses are Li and L[. The RG equations 
of (/3 and g' 3 will be mixed with gi and g[ through the 
diagrams in Fig. [5] The final coupled RG equations are 
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The perturbation with the highest scaling dimension at 
the fixed point with = is 



A = -3/201- l/2^+3 53 + 9' 3 , 



(17) 



the scaling dimension is — e + 64/(4A7r 2 ). When N < 
N C 2 = 64/(47r 2 e) coupling constant A is clearly rele- 
vant, but when N = 2 at the first order calculation of 
1/N expansion all the four fermion terms are irrelevant, 
the highest scaling dimension is about -0.189. However, 
higher order 1/7V 2 corrections might change this result. 



The critical N& we calculated is consistent with the pre- 
vious calculations in the context of spontaneous chiral 
symmetry breaking mass generation of QED 3 18 ' 2 b 22 i 23 . 

Now let us assume N < N c2 , and after a long RG 
flow all the irrelevant couplings are negligible. Thus at 
low energy and long wavelength, g% ~ g' 3 , g\ — g[ ~ 0, 
and the relevant coupling A = Ag^ — Ag 3 . Based on 
equation l|15jl . positive relevant A tends to favor SU(2N) 
symmetry breaking order (V^sufaN)'^) ^ ^' ana - negative 
relevant A tends to favor SU(2) symmetry breaking order 
{ippLtjj) 0, which is usually referred to as chiral symme- 
try breaking mass generation. In this case the SU(4N) 
symmetric spin liquid becomes a critical point between 
two phases with different symmetry breaking. 

As is shown in the appendix, with the presence of all 
the four fermion terms considered so far, the symmetry of 
the system is broken down to Sp(2N) x U(l), mainly be- 
cause neither the SU(2N) group nor the SU(2) group is a 
subgroup of Sp(4N). In the case of AT = 1 and staggered 
flux state, Sp(2N) subgroup is the SU(2) spin symme- 
try, and U(l) is the effective 0(2) rotation of the planar 
vector formed by VBS order parameters. In the case of 
N = 2 and 7r— flux state of SU(4) spin system, realized 
in spin-3/2 cold atoms, Sp(2N) subgroup is the unfinc- 
tuned Sp(4) pseudospin symmetry. 

At the first order l/N expansion, two parameters 
have equally the highest scaling dimensions: g 2 and 
A = — 3/2<7i — l/2fi4 + 3<73 + g' s , but the equality be- 
tween the two scaling dimensions are not protected by 
any symmetry. If N < N c = Min[A c i, N&], both g 2 and 
A are relevant, and g 2 is likely to drive the system to 
a fixed point with Sp(4N) symmetry. Now let us focus 
on the vicinity of this Sp(4N) fixed point. If we take e 
of order unity, the correction of the scaling dimension 
from fixed point value g% will be at order l/N, and L3, 
L' 3 will be mixing with many other terms with symmetry 
Sp(2N) ® U(l), the RG equations are rather complicated, 
but it is very unlikely that there is no relevant flowing 
eigenvector. Without detailed RG calculations, many re- 
sults can be obtained intuitively. Based on the identity 
(|45[) proved in the appendix we have: 

2JV(4iV+l) 

£ (^sp(4N)^)(^ p(4N ^) = 
a=l 

~2NJ a ^J l3c7 (^ a ipf3)(i(;^ a )-\ , 

8JV 2 -27V-1 

(^su(4N)/sp(4N)V')(V'T 1 S u(4N)/sp(4N)' ( /') = 

a=l 

2NJ ai Jf3 a {i) a ijp){4>^ a ) + -- - (18) 

As was discussed in the previous paragraph, without 172, 
relevant A tends to favor either SU(2N) symmetry break- 
ing order (^T^^ip) ^ or SU(2) symmetry break- 
ing order {ippiip) ^ 0, depending on the sign of A. No- 
tice that subalgebra Sp(2N) ® 1 and 1 ® ^ z belong to 
Sp(4N), while SU(2N)/Sp(2N) and 1 ® fi x , 1 ® n y all 
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FIG. 5: The RG flow when N < N c = Mm{N cl ,N c2 ], the 
horizontal axis is 32, and the vertical axis is A in (|17l) . At 
the Sp(4N) fixed point the order parameters with equally 
strongest correlations are i)T^ 2 N)' t l ) and , ipfj, z ip, thus phase 
A is most likely to be order {tpT^ p ^ 2 N) VOj an d phase B is most 
likely to be {tp/j, z 4>}- Notice that A is an RG eigenvector at 
the SU(4N) fixed point but not at the Sp(4N) fixed point, but 
phase A and B can be driven by A perturbation at the Sp(4N) 
fixed point. 



belong to SU(4N)/Sp(4N). A positive g 2 will favor or- 
der (^T s ° p(2NI V) over (^'C ( 2N)/sp(2N)V'} when A > 0; and 
also favors {ipii z ip) over (i/j^ip), (ip/jPip) with negative A. 
Equation (fTTj) also shows that order parameter V^^N)^ 
and i^/j, z tp have stronger correlation and hence stronger 
tendency to order at the Sp(4N) fixed point compared 
with '0T s ( ^ 2N) / sp( - 2N )i/' and ■tp^ x ip, -tp^tfj. Therefore the 
Sp(4N) fixed point is a critical point between Sp(2N) 
symmetry breaking order (^pT^^^ip) and order (tp^ip). 
The RG flow diagram is shown in Fig. [5] 

From the first order 1/N expansion, N c is probably 
larger than 1. In the case of staggered flux state with 
N = 1, the theories above show that with the four 
fermion terms considered so far, the Sp(4N) fixed point 
is a critical point between a SU(2) symmetry breaking 
state, and a state which breaks time reversal symmetry, 
but no spin or lattice translational symmetry is broken. 
However, the SU(2) symmetry breaking state is not the 
Neel order. The physical interpretations of these fermion 
bilinear order parameters can be found in reference^. If 
the critical number N c is also greater than 2, as in the 
case of SU(4) tt— flux state, the theory describes a critical 
point with Sp(8) symmetry between an SO (5) symmetry 
breaking phase with order parameter ipTabip, and a stag- 
gered chiral state which breaks translational symmetry 
and time reversal symmetry. Here r a f, = ^[r^r;,] are 
spinor representations of SO (5) (Sp(4)) group, and r a (a 
= 1, 2 • • ■ 5) are five Gamma matrices. 

When any of the fermion bilinear order is developed in 
the system, the fermionic spectrum is gapped. In the case 
of gapped matter field, the compact nature of the U(l) 
gauge boson is no longer negligible, and the monopole 
proliferation usually opens a gap for the gauge boson, and 
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confine the gapped matter field. However, if the gapped 
fermions form a topological insulator, the U(l) gauge 
field is not necessarily confining. For instance, consider 
a Dirac fermion system with conserved fermion charge 
coupled with a compact U(l) gauge field, if the fermion 
gap ipip is turned on, the system enters the quantum Hall 
state of spinons, and the Hall conductivity is half of the 
number of Dirac nodes. A Chern-Simons term is gener- 
ated for the compact U(l) gauge field, in which case the 
monopole effect is suppressed 2 ^. This result can be un- 
derstood physically as following: a monopole in 2+1 di- 
mensional space-time annihilates/creates 2tt flux of gauge 
field; however, an adiabatically inserted 2tt gauge flux 
will trap one spinon due to the quantum Hall physics. 
And because of the conservation of the spinon number, 
the 2-7T flux cannot be created or annihilated freely. 

In the order (tpT^^^ip) , the sign of the fermion gap, 
i.e. the sign of the Hall conductivity depends on the 
Sp(2N) spin component. If a 2ir— flux is adiabatically 
inserted in the system, it will trap nonzero charge of 
^sp(2N) ■ ln the past few years the quantum spin Hall 
effect (QSHE) has attracted a lot of attention, and 
many versions of QSHE models have been proposed, 
most of which are two copies of quantum Hall states 
with opposite Hall conductivities for spin up and down 
component s 25 ' 26 . Very recently the QSHE has been ob- 
served in experiment o 27 ' 28 . In our case the state with 
(^sp(2N)'^) ^ s actually a Sp(2N) generalization of a quan- 
tum spin Hall model coupled with a compact U(l) gauge 
field. A nonzero Sp(2N) spin will be trapped by an adia- 
batically inserted 2tt gauge flux due to the QSHE effect. 
Because of spin conservation, the monopole effect is again 
suppressed, thus in this state the spinons are gapped but 
not confined. However, the stability of the spin-filter edge 
states against the gapless U(l) gauge boson in the bulk 
requires more careful analysis. This type of states will 
be studied carefully in future 2 ^,. In the order ('0/j, 2: 'i/7) , 
fermion gaps are opened for two Dirac valleys with op- 
posite signs, i.e. the total charge Hall effect is zero. Also, 
since the 0(2) rotation symmetry of fi x and \i v is broken 
down to on the lattice, ^ is not precisely a conserved 
quantity. Therefore the flux tunnelling is allowed, the 
monopoles are not suppressed, and the spinons are still 
confined due to the compact nature of the U(l) gauge 
field. 

We want to point out that we have not yet ex- 
hausted all the four fermion terms allowed by the phys- 
ical symmetry Sp(2N) x U(l). Terms like (V'aaMa&V'ab) 2 , 
JajJpA' l Paatpi3a)('4'-fb^crb) and some others are all al- 
lowed. Here J' is the antisymmetric tensor of the 
Sp(2N) algebra, and in the appendix we will prove that 
3 = 3' ® [i x ■ Different four fermion terms will favor dif- 
ferent ordered patterns. For instance, if Sp(2N) x U(l) 
perturbation £ b (^T fc p(2N) ®^ z ^f -ELsW'W) 2 flows 
to the cut-off energy scale, it will drive a phase transition 
between the Sp(2N) Neel and VBS order. 



III. SPIN LIQUID WITH SU(2) GAUGE FIELD 

A. The Majorana fermion formalism 

The 7T— flux state of SU(2) spin system on the square 
lattice enjoys SU(2) local gauge symmetries. The low 
energy effective theory of this state is 

2 3 

i = EE^^- w >U + '- (is) 

a=l 1=1 

On the lattice, the variational parameter Uij hopping 
matrix is chosen to be: fT^+a = (— l) y iT° , Ui^+y = ir , 
and the 2-site unit cell is chosen to be (i,i + y). For this 
choice of gauge, the Dirac points are located at (0, tt/2) 
and (n,ir/2). We use a and b to denote this two Dirac 
node valleys. The Dirac gamma matrices are: 70 = <r 2 , 
71 = — cr 1 , 72 = — o" 3 . It is believed that when the fermion 
number is large enough, action (|19[) describes a conformal 
field theory; when fermion number is small, the system 
is instable against confinement due to the antiscreening 
interaction between SU(2) gauge boson o 30 ' 31 . In this sec- 
tion we will discuss another type of instability of this con- 
formal field theory driven by four fermion interactions. 

The SU(2) gauge field is operating on^= (i/>i, ^2) T = 
(/l,— /|) T . However, physical spin SU(2) symmetry is 
not obvious in action (| 19[) . Since the charge density in 
equation (fT!))) is actually spin density S z , the charge cur- 
rent "07m V' is n °t a singlet under spin SU(2) transforma- 
tion. In order to resolve this problem, reference^ enlarged 
the fermion space. However, after this treatment there 
is a constraint on the fermionic space: ip* and ip are re- 
lated through a unitary transformation, this makes the 
calculations based on action (fT9)) inconvenient. In this 
section we will first introduce a Majorana fermion for- 
malism for the 7T— flux state. In this Majorana fermion 
formalism both SU(2) gauge symmetry and SU(2) spin 
symmetry are both apparent, and there is no constraint 
on the fermion multiplet. 

We define 8 component Majorana fermion multiplet x : 



Xm 


= Re('01a), 


X211 


= Im(V>i a 


X121 


= Re(V>2a), 


X221 


= lm(0 2a 


X112 


= Re^ib), 


X212 


= Im(V>i6 


X122 


= Re(V>2b), 


X222 


= Im(V>26 



); (20) 



Each index of \ denotes a two-component space. The 
pauli matrices operating on the first, second and third 
two component space are denoted by r a , er a and \x a 
respectively. If we ignore gauge fluctuations, this sys- 
tem enjoys an SO (8) symmetry, and all the symmetry 
transformations including the SU(2) gauge transforma- 
tions are subgroups of this SO(8) group. We are going to 
write all the physical order parameters in terms of bilin- 
ears of the Majorana fermion x^Xi the fermion statistics 
requires matrix T to be antisymmetric. 
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Now we try to reformulate this theory in terms of x- 
In the Majorana fermion space, the three SU(2) ma- 
trices are 

G 3 = r 2 <g> a x ® 1, 
Gi = r 2 <g> <T Z <g> 1, 

G 2 = -1 <8> cr y 1. (21) 

One can check that these three matrices, though mixing 
two different spaces, still form an SU(2) algebra. All the 
physical symmetry transformation should commute with 
this SU(2) algebra. 

The bosonic version of our formalism actually realizes 
the beautiful second Hopf map. One way to study the 
0(5) Nonlinear Sigma model is to decompose the 0(5) 
vector in terms of bosonic SU(4) spinors as n a = $Tp a $ ; 
T a with a = 1, 2, • • • 5 are five Gamma matrices, and $ 
is a four component complex bosonic spinor— . After this 
decomposition there is a redundant SU(2) gauge degrees 
of freedom, and since the four component complex bo- 
son $ contains eight real components, the effective field 
theory of 0(5) Nonlinear Sigma model with the Hopf 
term can be viewed as an 0(8) sigma model coupled with 
SU(2) gauge field. With unit length constraint, the 0(8) 
vector forms a manifold of seven dimensional sphere S 7 , 
and the theory describes a mapping: S 7 /S 3 — > S 4 , the 

5 3 manifold is exactly the SU(2) group manifold, and 

5 4 is the manifold formed by 0(5) vector. This is a di- 
rect generalization of the first Hopf map which gives the 
CP(1) model, which is a popular way of rewriting the 
0(3) Nonlinear Sigma model 9 . This second Hopf map 
has been used to construct the 4 dimensional quantum 
Hall fluid 33 . The Wess-Zumino-Witten term of the 0(5) 
Nonliear sigma model can also be derived from 2+1 di- 
mensional Dirac fermion actional. 

Inspired by the second Hopf map, the flavor symmetry 
of our theory should be SO (5), in which the spin rotation 
symmetry should be contained. After some algebras, one 
can see that the spin transformation SU(2) algebra is 

S z = -t 2 ® 1 ® 1, 
S x = t 1 ® a y <g> 1, 

S y = t 3 $ a y <g) 1. (22) 

It is straightforward to check that [S a ,Gb] = 0. The 
gauge group generators in equation (|2ip and spin rota- 
tion generators in equation (|22| together form an SO (4) 
algebra. 

There are in total 10 elements in the S0(8) algebra 
which commute with the SU(2) gauge algebra, they are 

S z = -t 2 <g> 1 ® 1, S* = t 1 <g> a y ® 1, 

S y = t 3 ® <r v ® 1; -t 2 <g 1 <g> n\ 



-t 2 <g> 1® fi x ; t 1 ® a y ®n z , 
t 1 (g>a y <g> fi x , t 3 <g>a y <g> n z , 

t 3 ®(t v ®^ x , 1 <g> 1 <g> m 3 '- (23) 

These matrices are all antisymmetric, and form an SO (5) 
algebra. Besides these antisymmetric matrices, there are 
five symmetric matrices which form an vector represen- 
tation of this SO (5) algebra: 

v x = t 1 ® <j v (g) fi y , r 2 = t 3 ® a v <gi n v , 
r 3 = -r 2 (g) i ig) [i y , r 4 = l ® l <gi n x , 

r 5 = i <g> i (g> n z . (24) 

The first three matrices form a vector representation of 
spin SU(2) group, and it can be checked that [G a , Ti\ = 
for all a and i. Now one can construct fermion bilin- 
ears with SO (5) algebra constructed in equation (T2"3"l) and 
the Gamma matrices constructed in equation The 
physical interpretation of all the bilinears are summarized 
as following 

Neel, n a : X T a X, a = 1,2, 3; 
ferromagnetic order, m a : xioS a X', 
VBS X : XM X X, VBS y : x^X, 
chirality : x,X\ 
staggered chirality : XloP v X'i 
{-l) x Si x S i+y : xioSy x x, 

{-l) v Si x S i+£ : xioSy z X - (25) 

In the above equation, x = X T 7o- These bilinears have 
exhausted all the elements in the SO (5) algebra and the 
T a matrices. All these fermion bilinears correspond to 
long wavelength fluctuations of certain order parameters 
on the lattice. The lattice version of spin chirality is 
Si • (5 2 x S±) + S 2 ■ (S 3 x Si) + S 3 • {S 4 x S 2 ) + S± ■ (Si x S 3 ), 
1,2,3 and 4 are sites on the four corners of a unit square, 
ordered clockwise. 

The mean field choice of Uij apparently breaks the 
lattice symmetry, thus the lattice symmetry transforma- 
tions should be combined with gauge transformations on 
the fermionic multiplet ip, which is usually called the pro- 
jective symmetry group (PSG). The complete PSG trans- 
formations combined with lattice symmetry are summa- 
rized as: 

T x : 1 <g> 1 <g)/z z , T y : 1 <g) 1 <g> fi x , 
P xs : 7i ®M X 5 p xb : 7i ® W V , 
P ys : 72 ® H z , Pyb : 72 ® i^ y , 
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(7i-7 2 )«>(M a; +^)/2, 



B. large-iV generalization and RG equations for 
four-fermion perturbations 



T : 70 <8> ia y <g> \i v ■ 



(26) 



T x and T y are translations, P xs and P ys are site cen- 
tered reflections, P x b and P y b are bond centered reflec- 
tions, and P xy is reflection along the line x = y, T is the 
time-reversal transformation. The time reversal trans- 
formation is an antiunitary operation, which transforms 
i — ► —%. Therefore as long as matrix T between xTx 
contains i, it always gains an extra minus sign under 
time-reversal. For all the fermion bilincars in equation 
(|25p . Neel order parameter, ferromagnetic order parame- 
ter, chirality and staggered chirality are odd under time- 
reversal; VBS order parameters and staggered triplet 
bond order (— l) 1 "^ x Si+p, are even. 

It is interesting to compare the fermion bilinear rep- 
resentations in the Majorana fermion formalism and the 
formalism in terms of tp. Introducing \& = (?/>, — i<T 2 ^*) T 
and ^> = as in reference^, the comparison between 

fermion bilinears in the x language and tp language is 
listed below: 

2xx = 2xr a x = *f 

X7a*X = *7/** = 0, xGaX = *o- a * = 0, 

xT a X = *T a * = 0. (27) 

G a with a = 1, 2, 3 are three matrices defined in equation 
(l2~Tj) . and T a are ten SO (5) algebra generators defined in 
equation (f2"3")l . Notice that ip are —ia 2 ip* both transform 
as spinors under gauge SU(2) group, a a with a = 1, 2, 
3 are three gauge SU(2) Pauli matrices. The spin SU(2) 
transformation will mix ip and —ia 2 ip*, the Dirac node 
valley space is another direct product space. T a with 
a = 1, 2, • ■ • 5 are five 4x4 Gamma matrices operating 
on the spin space and Dirac node valley space: 



~3 



(28) 



and T a with a — 1, 2, • • • 10 are fundamental representa- 
tions of ten 4x4 Sp(4) ~ SO(5) generators, which are also 
the commutators of T a matrices. Here a a with a = 1,2,3 
are three spin SU(2) Pauli matrices which mix if> and 
—ia 2 ip*', /i a with a — x,y,z are three Pauli matrices op- 
erating on the Dirac node valley space. 

Now the field theory of it— flux state in terms of Majo- 
rana fermions can be written as 



3 

£: 
i=i 



L =y > ~X7m( 9 p - ia uGi)x + 



(29) 



Here x = X T 7o- The ellipses should include all the four 
fermion terms allowed by PSG. 



The large-iV generalization of this problem can be 
achieved by increasing two-component fermionic spaces. 
The gauge field always only involves the first two 2- 
component spaces, and the gauge group is always SU(2). 
The details of large- N generalization is in the appendix. 
Basically, for n two-component fermionic spaces, the 
number of Majorana fermions is Nf — 2™, and the fla- 
vor symmetry which commute with the SU(2) gauge al- 
gebra is Sp(4N) with N = 2™ -3 . All the matrices in 
the particular representation of the Sp(4N) algebra are 
antisymmetric, and there are 8A 2 — 2N — 1 fermion bi- 
linears xTaX which form a representation of Sp(4N) al- 
gebra, T a are symmetric matrices. In the appendix we 
also proved that our large- N generalization corresponds 
to the 7T— flux state of Sp(2N) spin system. 

At the conformal field theory fixed point, the Majorana 
fermion propagators are 



{Xi,kXj,-k) - Oij . 



(30) 



This can be viewed as "half fermion propagator. The 
dressed gauge field propagator after integrating out the 
fermions is 



«(?Kh)> = i|-(V-^). 



N f q 



(31) 



In our physical situation Nf — 8. The scaling dimensions 
of some fermion bilinears can be calculated readily: 



A(xx) = 2 - 
A(xFaX) = 2 



256 

128 

n 2 N f : 



A(X7^°X) = 2- 



(32) 



For all the Majorana fermion bilinears, the matrix be- 
tween x should be antisymmetric. One thing worth no- 
tice is that, the SU(2) gauge current Xln^aX 1S 110 longer 
gauge invariant and hence has no well-defined scaling di- 
mension. On the contrary, the scaling dimension of SU(2) 
gauge singlet current XI ^ a X gains no 1/Nf corrections. 

The four fermion terms in the field theory should be 
invariant under all the symmetry transformations, thus 
they should be mixed under RG flow. The simplest four 
fermion terms are squares of Sp(4N) scalar fermion bi- 
linears. To identify all the terms of this kind, we need 
to find a symmetric tensor T or antisymmetric tensor 
J : which commute with gauge matrices G a and all the 
Sp(4N) flavor matrices. If these tensors exist, one can 
write down four fermion terms like 



]T(X7^G a x) 2 , £&7G°X) S 



(33) 



a=l 
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In the physical case with N = 1, the only symmetric 
tensor T one can find is the unit matrix, and there is 
no satisfactory antisymmetric J . The representation of 
SO (5) in equation (|2"3"|) belongs to a vector representation 
of SO (8) group and hence reducible, i.e. There are non- 
unit matrices commuting with all the matrices in (|23|) . 
However, the gauge invariance criterion guarantees only 
the unit matrix T is satisfactory. Therefore the only two 
linear independent four fermion terms of this type are 



ii = 



9i 
N f A 



(XX) 2 , L[ = 



\- 9i 



^ N f A 

a=l 1 



{xi^Gaxf. (34) 



In the original ip language, these two terms are (ipi>) 2 an( i 
; (^7 M cr' - 0) 2 . Gauge singlet Current-current interac- 
tion {xinX) 2 is n °t allowed because of fermion statis- 
tics of Xj i- e - m this theory there is no extra global 
U(l) symmetry. Both L\ and L[ are invariant under 
SU(2) gauge (g) Sp(2N) group, and they are mixed under 
RG flow at the linear order, i.e. the corrections from 
gauge field fluctuations. The Feynman diagrams that 
contribute to the anomalous dimensions are the same as 
the U(l) spin liquid case. The coupled RG equations for 
gi and g[ are 



d\nl 



512 



)9i 



384 , 1 2 



dg[ 

d\nl 



= (-e- 



1024 



)g[ 



128 



.91 



1 

3^ 



(35) 



At the conformal field theory fixed point, the most rel- 
evant combination is Ai = 0.44gi + g[, with scaling di- 
mension — e + 36.5/Nf, with critical Nf c i = 36.5, which 
is much higher than the spin liquids with U(l) gauge 
field fluctuation considered in section II. At the physi- 
cal case with Nf = 8, this conformal field fixed point is 
very instable, and no stable fixed point is found at finite 
four fermion couplings. The irrelevant RG flow eigen- 
vector is — 20.4<?i + g[, thus after long enough RG flow, 
g' x ~ 20.4<?i, i.e. L[ will dominate L\ at low energy 
and long wavelength, thus the phase driven by these four 
fermion terms prefers to minimize L[. L[ is a SU(2) 
gauge current interaction, and gauge current X7 M G a x is 
not gauge invariant. Therefore the order driven by L'-y can 
break the SU(2) gauge symmetry. For instance, if the rel- 
evant flowing eigenvector Ai is negative, it will flow to a 
state which spontaneously generates a finite SU(2) gauge 
current on the lattice scale, and this gauge current will 
break the SU(2) gauge symmetry down to smaller gauge 
symmetries. If Ai > 0, the possible state driven by Ai is 
a SU(2) gauge singlet fermion paired state. Therefore if 
the Major ana fermion number N is decreased from large 
enough value, two different instabilities will compete: the 
SU(2) gauge boson confinement tends to drive the sys- 
tem to an SU(2) gauge singlet ground state (the nature 
of this phase is not clear); while the four fermion inter- 
action studied in the current work can drive the system 
to a state with broken SU(2) gauge symmetry. 



When any fermion bilinear order (xTx) is developed, 
the fermion spectrum is gapped. The screening of gapped 
fermions can no longer overcome the interactions between 
SU(2) gauge bosons, the gauge coupling flow will confine 
all the excitations with nonzero SU(2) gauge charge, all 
the excitations of this phase have to be SU(2) gauge sin- 
glet. However, if the SU(2) gauge symmetry is broken 
spontaneously by the relevant four fermion terms, the 
residual gauge field fluctuation may or may not be con- 
fining, depending on the gauge group. If the residual 
gauge group is Z 2 , the gapped spinons can be still de- 
confined; if the residual gauge group is U(l), for instance 
when a uniform gauge current Xl^G a x is generated, the 
monopole fluctuation will confine the spinons, and the 
specific ground state order pattern is determined by the 
quantum number of monopoles. Notice that although 
both L\ and L[ are SO(5) invariant, the SO(5) symme- 
try can be broken by the quantum number of proliferat- 
ing monopoles. A full analysis of the monopole quantum 
number is not yet accomplished. 

In the appendix we showed that the SU(2) gauge in- 
variant formalism is only exact for Sp(2N) Hamiltonian 
with J2 = in equation ([53)) . When J2 ^ the system 
only enjoys the U(l) gauge symmetry, and if J 2 — J\ the 
spin model becomes SU(2N) invariant, and the 7r— flux 
state is described by QED3. Now let us consider turn- 
ing on a small J2 perturbation on the 7r— flux state of 
Sp(2N) spin Hamiltonian with only Jy in equation (|55[) . 
this perturbation will generate four-fermion perturbation 
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{2( X ^G 3 x) 2 - {xi^xf - ( X 7mG 2 x) 2 X-36) 



L 2 is one component of the d— wave vector under gauge 
SU(2) group. Since L 2 belongs to a different represen- 
tation under SU(2) gauge group from Ly and Ly, the 
linear RG equation of L 2 will not be mixed with Ly and 
L'-y. Also since xG a X vanishes due to fermion statistics, 
L 2 itself is an eigenvector under linearized RG flow to 
the first order of l/N expansion. 
The RG equation for L 2 reads 



dg2 
dlnZ 



(-e + 



256 
3tt 2 A7 



)92 



1 

3^ 



(37) 



Now the situation is similar to the L 2 considered in the 
U(l) spin liquid case. The scaling dimension of g 2 is 
-e+256/(37r 2 iV / ), and for N < N f , c2 = 256/(3tt 2 ) = 8.7, 
L 2 will drive the system to a fixed point with a finite <?2- 
At the fixed point of finite g 2 , the SU(2) gauge symme- 
try is broken down to U(l) gauge symmetry generated by 
G 3 , thus this fixed point is very analogous to the fixed 
point with finite L 2 discussed in the section II. The crit- 
ical value of Nf iC2 from the first order 1/Nf expansion is 
slightly larger than 8, and for the 7r-flux state of the Sp(4) 
spin model with Nf = 16, L 2 will not bring an instabil- 
ity to the state, and the finite 172 fixed point becomes a 
critical point. 

So far we have preserved the Sp(2N) flavor symme- 
try, which is larger than the physical symmetry. As is 
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discussed in the appendix, our large- TV generalization is 
applicable to the tt— flux state of Sp(2N) spin model with 
TV = 2"~ 1 . Therefore four fermion terms which break the 
emergent flavor symmetry down to physical symmetries 
certainly exist in the field theory. Let us assume the total 
number of 2-componcnt fermion space is k + 1, two terms 
of this kind are 



were conjectured in this paper, but more detailed cal- 
culation is required to determine which phases are most 
favorable ones. 



U = 



93 



93 



N 



Another physical system with low energy Dirac fermion 
excitations is graphene, where the Dirac nodes locate at 
the corners of the Brillouin zone. There are two flavors of 
Dirac fermions coming from the two incquivalent corners 
of the Brillouin zone, and another two flavors from the 
spin degeneracy. Thus in this system the total number of 
a Dirac fermions is N — 4. The difference between this case 

{2(xTfc ® /i y 7 A1 G b x) 2 - ^2 (x^l^x) 2 ]}- and our s P in liquids is that, there is no fluctuating gauge 
i =x , z field in graphene, except for a static Coulomb interaction. 

(38pue to the apparent Lorentz symmetry breaking of the 
Coulomb interaction, the fermi velocity will flow under 
RG. The effects of four fermion terms in graphene have 
been studied in reference 3 -^ 



Notice that fermion bilinear ® /i v x is the large-TV 
analogue of the Neel order parameter, x/J, x x an d XP^X 
are the large- N analogue of the VBS order parameters, 
therefore a relevant L3 will favor either Neel or VBS 
phase depending on the sign. The coupled linear RG 
equations for g 3 and g' 3 read 



dg 3 
din I 

dg' 3 



= (-e 



256 

K 2 Nf 



)93 



d\nl 



= (-*■ 



256 



f 



)93 + 



384 

' TT 2 Nf 

128 
3iT 2 N f 



93 ■ 



(39) 



The most relevant eigenvalue of the RG flow is — e + 
38.9/Nf, the critical value of Nf yC3 is 38.9, which is 
slightly higher than the critical value of Nf :C \ for L\ 
and L[ based on our first order 1/iV expansion. If Nf^ c3 
is indeed higher than iV/ jC ii when Nf >c i < Nf < Nf^ c3 
the tt— flux state is a critical point between Neel order 
xTr ® V> v X an d VBS order. The classification of the four- 
fcrmion terms is worked on elsewhere 3 -^. 



IV. CONCLUDING REMARKS 

In this work we studied the effects of four fermion in- 
teractions as one type of instabilities on several inter- 
esting algebraic spin liquids. The RG calculations show 
the gauge field fluctuation will generally enhance the rele- 
vance of the four fermion interactions, except for one par- 
ticular pair which preserves the SU(4N) emergent flavor 
symmetry in the spin liquids with U(l) gauge field. For 
the N = 1 U(l) spin liquid, several four fermion terms are 
relevant at the spin liquid. The four fermion term which 
breaks the SU(4N) symmetry to Sp(4N) symmetry will 
likely drive the system to a fixed point with finite cou- 
pling which describes a spin liquid with Sp(4N) symme- 
try, which is a critical point between phases with smaller 
symmetries. For the N — 2 case, all the four fermion 
terms are irrelevant at the first order 1 /N correction. The 
it— flux state with SU(2) gauge field is more vulnerable 
against four fermion terms, the critical fermion number 
is much higher compared with U(l) spin liquids. The 
specific phases driven by relevant four fermion couplings 



It has been suggested that the deconfine critical 
point between the Neel and VBS is of enlarged SO (5) 
symmetr y 37 ' 38 ' 39 , and the Neel and VBS order param- 
eters together form an SO (5) vector—. The deconfincd 
critical point between the Neel and VBS order is con- 
jectured to be a liquid phase of 0(5) Nonlinear sigma 
model with a Wess-Zumino-Witten term. A liquid phase 
with enlarged SO (5) symmetry can exclude many possi- 
ble relevant perturbations. In our theory, SO (5) sym- 
metry has appeared here and there, and both in the 
U(l) spin liquids and the SU(2) spin liquid the Neel and 
VBS order parameters form a five component S0(5) vec- 
tor. Although we have not completely identified the de- 
confine critical point in our theory, our formalism espe- 
cially the Majorana fermion formalism of SU(2) tt— flux 
state is still a promising approach to locate the decon- 
fine critical point, simply because of the beautiful second 
Hopf map. To do this, one needs to find a fixed point 
with Sp(4) flavor symmetry and only one relevant four 
fermion interaction which breaks Sp(4) symmetry down 
to SU(2) ®U(1). The fixed point with Sp(4) symmetry 
we identified in the U(l) spin liquid section has one ex- 
tra U(l) gauge symmetry compared with the 0(5) Non- 
linear sigma model description of the deconfined critical 
point 3 - 7 - , which in the dual language corresponds to the 
conservation of gauge flux. 

It is interesting to generalize the field theory of the 
deconfined critical point to larger spin systems, and 
one can approach these deconfined critical points from 
large- iV version of the spin liquids studied in this work. 
First of all, the VBS order can be naturally generalized 
to systems with Sp(2N) symmetry simply because two 
Sp(2N) particles with fundamental representation can 
form a Sp(2N) singlet through antisymmetric matrix J: 
Ja^X^^- The Neel order parameter spans an adjoint 
representation of the Sp(2N) group. The large- V formal- 
ism of spin liquids in our current paper shows that the 
smallest simple group with Sp(2N) ® U(l) subgroup is 
Sp(4N). Therefore if an unfine-tuned second order tran- 
sition between Sp(2N) Neel and VBS order exists, this 
critical point can enjoy enlarged Sp(4N) symmetry. 
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APPENDIX 



(44) 



A. Construction of fundamental representations of 
Sp(4N) algebra with N — 2 n 

In this appendix we will construct the fundamental 
representations of SU(4N) and Sp(4N) algebras with N = 
2". All the results will be proved by induction, thus we 
will first present all the results, which are obviously true 
for n = 0; later we will assume they are also valid for n = 
k, the same results for n = k + I can be proved directly 
from our construction of SU(4N) and Sp(4N) algebras. 

1st, SU(4N) algebra contains subalgebra 
SU(2N) <g> SU(2), the whole fundamental represen- 
tation of SU(4N) algebra can be constructed from the 
fundamental representations of its SU(2N) subalgebra 
and SU(2) subalgebra. All the SU(4N) algebra elements 
can be written as 



T a (g>//, T a ®l, l®/i 8 



(40) 



T a with a = 1, 2 • ■ ■ (2N) 2 — 1 are fundamental repre- 
sentations of all the elements in SU(2N) algebra, and // 
with i = 1 2, 3 are three SU(2) Pauli matrices. 

2nd, SU(2N) algebra has an Sp(2N) subalgebra, which 
satisfy 



ST rria rr /r ra \t 

^2NJ sp (2N)>-'2N - (,J sp (2N)J • 



(41) 



Here J72N is a 2N x 2N antisymmetric matrix. 

3rd, all the SU(2N) elements in SU(2N)/Sp(2N) sat- 
isfy 



ST rria 

J2NJ s11 



(2N)/sp(2N)^2N 



V su 



(2N)/sp(2N) 



(42) 



4th, all the elements in SU(2N)/Sp(2N) form a repre- 
sentation of Sp(2N); or more precisely, fermion bilinear 
^^su(2N)/sp(2N)^ s P ans a representation of Sp(2N) alge- 
bra. To prove this, one has to show that 

Kp (2N) , T s b u(2N)/sp(2N) ] e SU(2N)/Sp(2N). (43) 

The meaning of the equation above is that, the commu- 
tator between any element in SU(2N)/Sp(2N) and any 
element in Sp(2N) belongs to SU(2N)/Sp(2N). 

5th, all the elements in SU(2N) algebra satisfy follow- 
ing relations: 



7 1 S p(2N)^ s b p(2N) ]eSp(2N), 

]»i2N)' 'mu -N)/sp(2N)] e SU(2N)/Sp(2N), 

GSp(2N), 

{^( 2N ),^ ( 2N)} G SU(2N)/Sp(2N), 

(2N)> ^su(2N)/sp(2N) } e Sp(2N), 



[rria rpb 

V 1 su(2N) /sp(2N) > 1 su(2N) /sp(2N) 



I su 



rjib 

(2N)/sp(2N)i J su(2N)/sp(2N) 



} G SU(2N)/S P (2N). 



6th, for the fundamental representations of SU(2N) 
and Sp(2N) algebras, following identities are satisfied: 

(2AT) 2 -l 

^ ^su(2N),a/3^su(2N),7CT = 2NS acr Sp 7 - 5 a p5 1<7 , 



o=l 



JV(2JV+1) 



p(2N),a/3^sp(2N),7o 



2jV -iV-l 



V T a 

/ j si; 



su(2N)/sp(2N),Q/3^su(2N)/sp(2N),7<T 



= N5 aa 5f 3l + NJ ai J fj(J - 5 a f } 5 1CT . (45) 

All these identities have been used in the main text of 
our paper. 

Now the Sp(4N) algebra which is a subalgebra of 
SU(4N) can be constructed as 



^sp(2N) ® M 31 ! ^sp(2N) 



1 ^sp(2N) 



1, 



1 <g> fj, , T S u(2N)/sp(2N) ® 



(46) 



There are in total 2N(4N+1) elements in (|4o]l. All these 
matrices satisfy 



rr rpa rr /' r 7~' a ^^ 

^4NJ sp (4N)>V4N — t, J sp(4N)J 

Jan — Jw ® H- x 



(47) 



Meanwhile, All the elements in SU(4N) constructed in 
(301 but not in Sp(4N) constructed in equation (jlo) are 

l<8>fi x , l®n v , 

r su(2N)/sp(2N) ® M^; r su(2N)/sp(2N) ® M^J 



(2N)/sp(2N) 



1- ^ p(2N) ®/i Z 



(48) 



There are in total 8N 2 — 2N — 1 elements in equation 



All the equations from (gSJ) to (gTJ) are valid for SU(4) 
and Sp(4) algebras. Let us assume these results are true 
for n = k, then for n = k + 1, equations (|47|) . (|42| . 
(H2J), (UU) and (|4"S")) can be checked directly through con- 
structions in equations (J3HJ) ■ 63) an d (|4"8|) , and by using 
the assumptions made for n = k. The calculations are 
tedious but straightforward. 

We have proved that the fundamental representation 
of SU(4N) algebra with N — 2" can all be constructed 
by Pauli matrices, thus {T® n , m s) 2 = 1- Because of this 

and equation (g3J), vector n a = ($Tj^,^ , r^ip) rotates 

under Sp(4N) group, while keeping the length J2a( na ) 2 
constant. 

One can see that the SU(2N) subalgebra of SU(4N) 
does not completely belong to Sp(4N) constructed in 
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equation (|46[) . Instead, only the Sp(2N) subalgebra is 
a subalgebra of Sp(4N), and the SU(2N)/Sp(2N) part 
belongs to SU(4N)/Sp(4N). Meanwhile, the subalgebra 
SU(2) which commute with SU(2N) is not a subalge- 
bra of Sp(4N) either, only element /i z which gener- 
ates U(l) rotation belongs to Sp(4N). Therefore when 
the SU(2N) (8 SU(2) and Sp(4N) four fermion terms 
both exist, the symmetry of the system is actually only 
Sp(2N)®U(l). 



B. Large-iV generalization of the n —flux state of 
SU(2) spin model 

In this subsection we will show that for Majorana 
fcrmions with n + 1 two-component space coupled with 
SU(2) gauge field, the flavor symmetry is Sp(4N) with 
N = 2 n ~ 2 . In our paper we showed that for n = 1 and 2, 
the flavor symmetry is SO(3)~Sp(2) and SO(5)~Sp(4) 
respectively, and for n = 2 there are 5 symmetric matri- 
ces which make fermion bilinears xTaX span a represen- 
tation of Sp(4). We will try to generalize these results to 
larger number n. For n — k, let us first denote the Sp(2N) 
algebra elements as T%, and denote the space spanned by 
the symmetric matrices as r sp ( 2 N),k; and second, assume 
for n = k, following algebra is valid: 

\n,T b k ] eS P (2N) fe , [r^r|]eS P (2N) fe , 
P^r*] e r sp ( 2N ) !fc , {T k ,T k } e r sp ( 2N ) :fc , 
{T a k ,r b k } e r sp(2N) , fc , {T fe a , r fc j e s P (2N) fe . (49) 

These algebras are valid for the simplest case with n — 1 
and n = 2. 

Now we construct Sp(2N) and r sp ( 2 N) for n = k + 1 as 
following: 

Sp(2N) fe+1 : 

r fc a ® M *, T£®»*, n®yy, z?®i, i® M y ; 

r S p(2N),fc+i : 

l®/x x , l®n z . (50) 

fi a are Pauli matrices in the new two component space. 
Although for this representation of Sp(2N) algebra there 
is no antisymmetric matrices J which satisfies JT a J = 
(T a )*, the construction in equation (|50[) is exactly the 
same as the construction in (|46[) in the previous section, 
except for exchanging fi z and fi y , thus the algebra in 
equation is Sp(2N). There are in total N(2N+ 1) el- 
ements in Sp(2N), and 2N 2 — N — 1 elements in r sp ( 2 N)- 
The validity of algebra in (|49p for n = k + 1 can be 
checked directly by using the assumption (|49p and con- 
struction ([50)) . Notice that all the Sp(2N) elements in 



this representation are antisymmetric and belong to a 
vector representation of a larger group SO(2 fe+1 ), and 
the fermion bilinear vector X-ToX rotates under Sp(2N) 
group, with invariant vector length. The Sp(2N) algebra 
constructed this way is the largest flavor symmetry com- 
muting with the SU(2) gauge algebra in equation (f2"Tj) . 

In our calculation we have generalized the SU(2) 
7T— flux state to the case with larger number of fermion 
flavors by increasing the number of 2-component fermion 
space. What kind of lattice model can the large- iV gen- 
eralization be applied to? Recall that the smallest spin 
group SU(2)~ Sp(2), one way of generalizing SU(2) spin 
system is to generalize the spin symmetry to Sp(2N), 
and let us assume N — 2™. The lattice spin Hamiltonian 
reads 

H = JS«S«, (51) 

«,3> 

S a are N(2N + 1) Sp(2N) Lie Algebra elements. Intro- 
ducing spinon f a in the usual way S a — fXT^^fp with 

half-filling constraint }} a fi, a — N, we can use the fun- 
damental representation constructed in equation (|46[) in 
the appendix to rewrite the Hamiltonian (|51[) as 

<i,j> 

The meanfield variational parameters are defined as 

Xij = (ftpfj,?), Vij = J a p{fi,afj,p)- (53) 

In the above Hamiltonian we have performed suitable 
transformation to make J = ia v ®\\%- ■ -(S>ln, with N = 
2 n . After particle- hole transformation, we define fermion 

multiplet Vi, a = {fx, ■ ' • Jn) t , ip2, a = ■ ■ • f\ N ) T ■ 

the meanfield Hamiltonian can be written as 

H = NJ{4^JJ l3 . ab ^ a + H.c. + 

<i,j> 

Uij = iRe(x) + Im(x)r 3 + Re(?7)r 1 + Im^rXBA) 

The Hamiltonian (|5~4"1) enjoys a same SU(2) local gauge 
symmetry as the SU(2) spin meanfield Hamiltonian 5 . 
The Sp(2N) generalization of the tt— flux state can also 
be found in references, where an opposite logic was 
taken, the Sp(2N) spin operators were constructed from 
fermionic spinons. 

The meanfield choice of variational parameters is the 
same as the SU(2) n— flux state: C^+a = (— l) v iT°, 
Ui.i+y — ir°, and the 2-site unit cell is chosen to be 
(i,i + y), the rest of the formulation is the same as the 
SU(2) spin case, and the SU(2) gauge symmetry is pre- 
served in the low energy field theory. The flavor symme- 
try of the low energy field theory action of the Sp(2N) 
tt— flux state without four-fermion terms should include 
the U(l) rotation between the two Dirac nodes. Using the 
results in appendix A, it is straightforward to show that 
the smallest simple group with Sp(2N) <E> U(l) subgroup 
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is Sp(4N) with N — 2™. Therefore our large- N general- 
ization is applicable to the tt— flux state of Sp(2N) spin 
system with N — 2™. 

The SU(2) gauge symmetry of ([54")) is only exact for 
Sp(2N) spin Hamiltonian (|5ip . However, equation (|5~lj) 
is not the only way to write down a nearest neighbor 
Sp(2N) Hamiltonian. The general Hamiltonian reads 

H= JiTiTf + J 2 T a i T a ] , 

<i,j> 

T a G Sp(2N), T a G SU(2N)/S P (2N). (55) 
The lattice SU(2) gauge symmetry is only exact when 



J 2 = 0. When J\ — J 2 the system enjoys the SU(2N) 
spin symmetry, and it is known that the it— flux state 
of SU(2N) system only has U(l) gauge symmetry when 
N > 1. Thus if we turn on a small J 2 perturbation at the 
tt— flux state, it will induce four-fcrmion terms breaking 
the SU(2) gauge symmetry. 
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